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The Center for Vocational and Technical Education has 
been established as an independent unit on The Ohio State 
University campus with a grant from the Division of 
Comprehensive and Vocational Education Research, U. S. 
Office of Education. It serves a catalytic role in 
establishing consortia to focus on relevant problems in 
vocational and technical education. The Center is 
comprehensive in its commitment and responsibility, 
multidisciplinary in its approach, and interins titutional 
in its program. 



The major objectives of The Center follow: 

1. To provide continuing reappraisal of the 
role and function of vocational and tech- 
nical education in our democratic society; 

2. To stimulate and strengthen state, regional, 
and national programs of applied research 
and development directed toward the solution 
of pressing problems in vocational and 
technical education; 

3. To encourage the development of research to 
improve vocational and technical education 
in institutions of higher education and 
other appropriate settings; 

4. To conduct research studies directed toward 
the development of new knowledge and new 
applications of existing knowledge in 
vocational and technical education; 

5. To upgrade vocational education leadership 
(state supervisors, teacher educators, 
research specialists, and others) through 
an advanced study and inservice education 
program ; 

6. To provide a national information retrieval, 
storage, and dissemination system for 
vocational and technical education linked 
with the Educational Resources Information 
Center located in the U. S. Office of 
Education. 
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PREFACE 



This publication is the report of the National Research Con- 
ference on Consumer and Homemaking Education which was held to 
assist individuals in implementing the charge of Part P in the 
Vocational Education Amendments of 1968. In fulfilling the obli- 
gations and opportunities inherent in consumer and homemaking 
education, research and development plays a viable role. It was 
the purpose of this conference to bring to bear a wide range of 
talents and resources in the identification of national-level re- 
search and development priority problems, and to identify ways 
and means through which the professional community could impact 
on these problems . 

In addition to this publication, individuals will be inter- 
ested in reading the publications which were developed concurrent 
with plans for the conference. Bibliography of Research on Con- 
sumer and Homemaking Education and the publication following the 
conference. Review and Synthesis of the Bibliography on Consumer 
and Homemaking Education. 

We acknowledge the following persons for their contributions 
in completing this report: Anna M. Gorman, project director, and 

A. J. Miller, coordinator for development and training. The Center 
for Vocational and Technical Education. We appreciate the efforts 
of the following persons who served as reviewers for this publica- 
tion: Beverly Crabtree, chairman of home economics education. 

University of Missouri; and Gwendolyn Newkirk, associate professor 
of home economics education, University of Minnesota. William 
Hull and Frank Pratzner, research specialists at The Center, also 
gave invaluable assistance as reviewers. 

Robert E. Taylor 

Director 

The Center for Vocational 
and Technical Education 
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INTRODUCTION 



With the passage of the Vocational Act of 1963 and the Voca- 
tional Education Amendments of 1968, many new challenges have 
emerged for vocational educators . Traditional programs have been 
given new directional emphases. Specific proportions or alloca- 
tions of funds have been designated for helping individuals who 
have learning and occupational problems. Programs are to be held 
accountable for the products they produce and this involves the 
evaluation of student learning. These and many other challenges 
have led to the need for a concerted national effort in funda- 
mental and applied research which focuses on critical problems 
inherent within these challenges. 



CONFERENCE OBJECTIVES 

Critical problems and issues challenge Consumer and Homemaking 
Education today and these require research for their solution. A 
national research conference was held to: 1) determine priorities 

for the critical problems facing Consumer and Homemaking Education 
and 2) plan research projects which focus on these critical prob- 
lem areas. More specifically the objectives for the research con- 
ference were: 

1. To identify critical research problem areas in Coixsumer 
and Homemaking Education that are of national significance 

2. To identify a priority list cf researchable problems. 

3. To describe research strategies, methodologies, and in- 
stitutional-state resources to contribute to the solution 
of critical problem areas . 

4 . To determine lines of coordination for completion of 
research projects. 

5. To disseminate the conference proceedings, including the 
proposed research projects, and assist in the dissemina- 
tion of the completed project proposals. 

The program for the research conference is contained in Appendix A 
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PART I 



CONFERENCE OBJECTIVES 1 AND 2: 



To identify critical research problem areas in 
Consumer and Homemaking Education that are 
of national significance. 

To identify a priority list of researchable problems. 



PART I 



PRE-CONFERENCE ACTIVITIES 

A tentative list of critical research problem areas in Con- 
sumer and Homemaking Education was developed by use of the DELPHI 
Technique (see Gray paper for description of the DELPHI Technique) . 
Eighteen individuals were selected to serve as experts in develop- 
ing the list (Appendix B) . The experts represented, at the least, 
seven areas: business education, consumer and family economics, 

distributive education, economic education, home economics, home 
economics education, and vocational education. 

Four questionnaires were developed by the project director to 
implement the concepts of the DELPHI Technique. The sequential- 
type questionnaires were completed from February to May 1970, by 
the experts (Appendix C) . 

CONFERENCE ACTIVITIES 
PRIORITY RESEARCH PROBLEM LIST 

Participants (Appendix D) received the tentative list of 
critical research problems (pp. 139-141) on Consumer and Homemak- 
ing Education in the registration packets. This list formed the 
background for the development of a finalized list by the confer- 
ence participants . 

SUMMARY OF COMMENTS RELATING TO THE 
"REAL WORLD OF THE CONSUMER" 

Three people were responsible for leading a discussion of the 
"Real World of the Consumer." The purpose of the discussion was 
to jar the participants into thinking and deciding upon priority 
problems that, in fact, were "real for people" and not just aca- 
demic and/or administratively interesting. A summary of their 
comments follows. 

Based upon data from 12 nutrition aides who work with 450 
families in Mahoning County, Ohio, Converse concluded that the 
problems encountered by families living in poverty as they relate 
to nutrition and food include: first, there is a lack of avail- 

ability of stores for purchasing of food. This problem was pivotal 
to the development of other needs, for example, the need for trans- 
portation and the need for more food stamps and/or money because 
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of higher prices associated with independent grocery store owner- 
ship. Second, food and money for food running out before the end 
of the month is another problem area. One of the educational 
problems is helping these families to do grocery list planning to 
include food to fill up the family as well as properly nourish 
them. Another educational problem is helping these families to 
know what is the best buy in the neighborhood store. 

Other problems associated with rural poverty families have to 
do with the language or educational barrier. They lack general 
education competencies such as use of the English language and 
mathematics. How is it possible for them to know what is a good 
buy without these basic skills? 

There are many more immediate concerns caused by lack of mod- 
ern conveniences facing these individuals. They do not have water 
in their houses. Water has to be carried for dishes and bathing, 
and many children stay home from school rather than attend when 
they know they are not as clean as the other children. It is much 
easier to buy prepared food because of lack of water and refrigera- 
tion, and it is difficult to buy in quantity because of these lacks 

Converse suggested the use of aides was of great assistance 
to her and to these families "for there are not enough of us (home 
economists) to help those in need." She said an aide, in order 
to be successful must be sincere, tactful, know when to go away 
when not wanted, and know how to give suggestions. She found that 
most families responded to the aide, if she (the aide) can adjust 
to their way. 

Rosenfield gave a general description of the level of poverty 
in the United States and of the inadequacy of funding under the 
present system of defining poverty. 

Before adequate legislation could be passed (legislators are 
trying to find answers to the poverty problem) , the following 
questions must be answered: 

1. How do welfare families spend food money? 

2. What do we know about our social programs? 

3. How do families cope with their problems? What are their 
choices? 

4. Do work incentives work? Do families need work incentives 

5. How do you help children escape poverty? 

Rosenfield stressed three interrelated problems of families 
living in poverty. They do not have enough money; there is 
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inadequate housing, and transportation is nonexistent. While there 
were many training programs there was not enough documentation of 
their impact. We have a tendency to "blame the poor for being 
poor." Rosenfield concluded that neither work alone nor money 
alone was an escape from poverty. 

Hendricks discussed what disadvantaged people wanted most, 
barriers to their getting them, and research needed. 

She felt that people living in poverty want most: 

a. A job they are able to perform. They want to earn their 
own way to fulfill their concept of being a worthy human 
being . 

b. independence. They need transportation before they 

can take advantage of day care centers, employment, and 
clinics. Before they can become economically independent, 
they need to know how to wisely budget the money they 
have; they need to have special considerations given to 
them, such as checks every two weeks while they are learn- 
ing to budget money. 

c. A place to live. 

Among the barriers to attaining the above goals were? 

1. Middle class standards 

2. Cultural standards 

3 . Communication 

4. Television advertisement 

5. Role concept 

6. Time for learning 

7. Poor image 

Hendricks concluded that research on each of the following 
problem areas was needed: 

1. Kind of jobs that fit their capabilities. 

2. How to strengthen role concepts. 

3. Type of teacher these people will accept. 

4. Type of instruction for them (low income people 
themselves) . 
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5. Type of instruction for teachers of people with 
low income including internships and/or scholar- 
ships. 



SUMMARY OP "PANEL DISCUSSION OP THE TENTATIVE 
LIST OP PRIORITY RESEARCH PROBLEMS" 

Four members of the DELPHI team of experts served as reactors 
to: 1) review for the participants the rationale used in making 

decisions regarding priority research problems on Consumer and 
Homemaking Education; and 2) react to the tentative list developed 
by sharpening up of problem statements, commenting on unresolved 
research problem areas, and relating to parallel research studies 
from other fields which should be done. The tentative list of 
priority research problems is in Appendix C. 

Allen suggested a background of social and economic problems 
of society which give rise to educational programs. The National 
Vocational Education Advisory Panel recomandations gave rise to 
the wage-earning aspect of home economics education in 1963 and 
the consumer education, disadvantaged, and dual-role emphases in 
1968. She characterized home economic:* education as a process 
where learnings are applied, practical, useful, continuous. 

The problem areas endorsed by Allen as being of prime impor- 
tance were as follows: 

The development of programs to impact on three major 
problems of our society: upgrading of household work- 

ers (two million of them) ; the development of day care 
centers for children (staffing and management) ; and 
adult programs in home economics education to assist 
welfare families in child care and resource management. 

Curriculum research and/or revision to take into ac- 
count the changing role of women in relation to the 
home and family (i.e., the dual-role). Such research 
should also take into account the inclusion of high 
school boys in home economics classes. 

From Questionnaire No. 2 (Appendix C) . Action re- 
search to find and/or develop ways of instituting 
curriculum changes at a more rapid pace. In today's 
world, it takes too long for the "best wo know to do 5 ' 
to be universally done. 

Bymers expressed concern that of the top 27 priority research 
problems dorived by use of the DELPHI Technique, 15 or 16 fell into 
"breaking, packaging, and standardization" techniques rather than 
being of a substantive nature. 
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Bymers then discussed some priorities which she believed were 
pertinent for the group. These were as follows: 

1. New look ac occupants' right- to- she Iter. We need to ex- 
plore ways disadvantaged families can achieve equity in 
shelter. 

2. Investment in housing , a social investment in health. 

3. Perception of money. How do people perceive money? 

4. Point-of-purchase consumer information. How do you get 
information to the point-of-purchase? What is the 
delivery system? 

5. How to "sell" entrance into household services without 
stigma attached. 

6. How to expedite curriculum change. 

7. How to make use of cooperative techniques in solving 
problems . 

Hurt stated that we need to concentrate our research efforts 
on focal points in home economics, education. We need to be sure 
that we have agreement on acceptable definitions of the terms we 
arc using. Among the list of problem areas suggested by the panel 
of experts, some can definitely be classified as problems in "con- 
sumer and homemaking education," but others are problems in "occu- 
pational homo economics," some "consumer education," some "family 
economics," and somo are "personal economics" problems. If we 
were discussing only problems in consumer and homemaking education, 
wa need to think in terms of problems related to instruction and 
curriculum in this particular aspect of the home economics field. 
Problems in the substantive areas would be appropriate for research 
by those in the subject matter content areas and not by home eco- 
nomics educators. A team approach could also be taken with home 
economics educators and subject matter content researchers selec- 
ting for study a problem which related to both fields and each 
selecting the aspect of a problem area appropriate to his particu- 
lar fiold. 

Consumer education is an important part of consumer and home- 
making education programs today. Research is needed on how to 
teach so poople become hotter consumers. She stated that the prob- 
lem of placement of content at different developmental levels is 
an important one. We need to define our terms here too. Are we 
teaching consumor education or consumer buying? 

In thinking of problems in consumer and homemaking education 
which need research, wo need to look at the total scope of the 
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program. How to teach nutrition education so people change their 
eating habits needs as much research as consumer education. What 
concepts are most important to teach in preparing for the dual- 
role also needs study. Our legislation asked us to do more about 
encouraging professional leadership. What factors contribute to- 
ward professional leadership development is a problem not on the 
list/ but does need study. 

Hurt remarked that another issue we need to resolve ar we 
think together is whether we want to identify research and/or 
developmental problems. How much attention should be given to 
basic research studies? To developmental studies? 



Further research is needed to find out what is the relation- 
ship between competence in home economics and occupational compe- 
tence in home economics and occupational competency. One study 
on the relationship between home environment and success on the 
job was conducted by Marguerite Scruggs at Iowa State University. 
We need to build on that study to learn to what extent home eco- 
nomics education can make a difference. 

She said another study needed is the identification of the 
basic competencies in homemaking and family life which, if stu- 
dents would attain by the time they graduate from high school, 
they would have some assurance of success in establishing and 
maintaining a home and family. If we had such a list identified, 
we could develop various strategies for teaching so we could be 
sure students had acquired the desired competencies. 

What to include in a curriculum and how to teach so disad- 
vantaged individuals can find a better life needed much further 
study. 



As we look ahead to the time when we have fewer socioeconom- 
ically disadvantaged individuals and families in our society, we 
need to consider the contribution which can be made by home eco- 
nomics. At the present time our services are much in demand but 
what will we be able to offer when most Americans are in the middle 
class? Or are affluent? Research is needed to help us be ready 
for expected changes in society so we can continue to make a worth- 
while contribution. 



Warmke endorsed the following related problems: 

a. Develop a comprehensively coherent conceptual framework 
which demonstrates how personal economic decision-making 
is related to aggregate economic activity and vice versa. 

b* Develop curriculum guides to implement the conceptual 
framework indicated in letter a. 
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c. Develop a selected bibliography of study materials that 
can be used by both teachers and students to implement 
the teaching of the conceptual framework indicated in 
letter a. 

d. Develop student materials to accompany the curriculum 
guides indicated in letter b. 

e. After sufficient teacher and student materials and strat- 
egies have been developed, conduct experimental research 
to compare the relative effectiveness of the various 
materials and approaches. 

Warmke reported that the National Personal Economics Committee 
had addressed itself to sub points a, b, c, and d (items listed in 
paragraph above) . The materials developed as a part of the Na- 
tional Personal Economics Project are available through the Joint 
Council on Economic Education. 

Warmke indicated a need for testing instruments on consumer 
competencies. He discussed a new test developed by the Psycholo- 
gical Corporation under the sponsorship of the Joint Council on 
Economic Education and stressed that an important feature of the 
instrument was that it related personal economic decision-making 
to the total economy and vice versa. 

Warmke enumerated three individual economic roles that might 
be examined as part of the home economics curriculum, via worker, 
consumer, and citizen. He stated that as a worker an individual 
is engaged in earning or receiving an income; the major activities 
of the consumer as a user of income include spending, borrowing, 
saving, and investing. As a citizen the individual influences 
collective decision-making and, therefore, influences the regula- 
tion and allocation of public versus private goods. Ho stated 
that the so roles and activities of the individual could form the 
content areas for personal economics which, in turn, could bo an- 
alyzed through a decision-making procoss that would include: 

1) identifying the activities an individual might choose to under- 
take, 2) examining the cost-benefit relationship of oach individ- 
ual consumer decision, and 3) determining the effects of the 
whole system on individual actions and the offocts of the in- 
dividual actions on the whole systom. 

Warmke closed his presentation by discussing a Study Materials 
Report prepared as a part of the National Porsonal Economics Proj- 
ect. The report is to supplement curriculum guides prepared for 
home economics, business education, and the social studies. The 
committee, Warmke stated, examined some 1700 individual study 
materials. Less than one out of 10 of the materials examined re- 
lated individual consumer decision-making to the total economy. 

He stressed the need for materials which go beyond buymanehip,. 




9 



SUMMARY OF DISCUSSION FOR 
FINALIZED PRIORITY PROBLEM LIST 



Eight email groups met after the panel discussion to: 1) 

evaluate the tentative priority research problem areas (Appendix 
C, pp. 139-141) in terms of their own insights in Consumer and 
Homemaking Education, and 2) suggest changes with the involved 
rationales in the priority of the problems. 

Working Notes Summary . At least three groups were concerned 
with the need for definitions , for example, research, research- 
able problem, consumer and homemaking education. Three groups 
were concerned with the type of research which was most pertinent 
to the needs— pure, and/or experimental, and/or developmental. 

It was noted that some groups were concerned with keeping the 
wage-earning aspect of home economics separate from consumer and 
homemaking education, unless related to the dual-role concept. 

One group suggested that it was necessary to identify perti- 
nent research criteria before identifying needs. Most groups 
determined that the parameters for the groups' recommendations 
included the scope of mandates or provisions in the 1968 Vocation- 
al Amendr'ents especially under Part F, Consumer and Homemaking 
Education. 



Agreements and Decisions Summary . The agreements and deci- 
sions' relateH - to - the - !T TentatIve"" Llit“of Priority Problem Areas” 
developed by the exports using the DELPHI Technique were noted 
first. 

The first problem area was reacted to by at least four of the 
groups. The original statement of the problem area follows. 

1. Comparative studies (in consumer and homomaking 
education) to determine: 

a. Effective ways of reaching out-of-school groups 
and poverty groups (methods, course patterns, 
facilities, etc.). 

b. Difference ±n trained and untrained workers 
(evaluation) . 

Ono group wanted to know in what ways do we define the people in- 
cluded. This same group was in general agreement that studies are 
needed in this problem area, but they suggested that consideration 
bo given to a broader scope. Another group restated the problem 
area as "Methods of working with poverty groups." The three sub- 
categories they suggested wore: 1) comparative study of methods, 

2) profile study of low incomo groups, and 3) study of behavioral 
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changes resulting from methods used. Another group wanted ’'in- 
school" added to the statement. The fourth group reacted by chang- 
ing the wording to read, "Comparative studies in consumer and 
homemaking education to determine effective ways of reaching out- 
of-schc^l groups and poverty groups (methods, course patterns, 
facilities, etc.) . 

The priority problem listed as number 2 (p. 139) was reacted 
to by at least three groups as noted in the agreements and decisions 
section. The original statement of the problem area follows. 

2. Identification of (consumer education) competencies 
which should be taught in the elementary school, the 
junior high, high school. 

One group suggested that consideration be given to developing a 
list of competencies in a hierarchy without identification of 
school level. Another group suggested that the problem area should 
be written as, "Identification of (consumer education) competencies 
which chould be taught at appropriate level of competency." An- 
other group suggested the following wording — "Identification of 
(consumer education) competencies which are appropriate for and 
of concern to students K thru adulthood." 

Number 3 (p. 139) on the Tentative List received only one 
written reaction from the recorders. The original statement of 
the problem area follows. 

3. Tho development of programs to impact on three 

major problems of our society: upgrading of 

housohcld workers (two million of them) , the 
development of day care centers for children 
(staffing and management) , and adult programs 
in home economics education to assist welfare 
families in child care and resource management. 

It was suggested that tho problem area statement be changed by 
deleting tho word "welfare" so that tho last part of the statement 
would r^ad "adult programs in homo economics education to assist 
families in child care and resourco management." 

Tho priority problem aroa listod as number 4 (p. 139) re- 
ceived three written comments from recorders. Tho original state- 
ment of tho problem area follows. 

4 . What changos in consumor bohavior can be attributed 
to instruction in educational programs? 

One group considered this a high priority item. Tho question raised 
by "this group was, "In what ways can wo moasuro tho impact of in- 
struction?" Another group docidod that number 4 would bo a 
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longitudinal study which would present a wide range of uncontrolled 
variables which would jeopardize its validity. Another group de- 
cided number 4 should be given further consideration. 



Some of the groups agreed with number 5 (p. 124) as it was 
stated. The original statement of the problem area follows. 

5. Determine tasks in teaching home economics (home- 
making- -consumer education) which can be performed 
by aides — persons with less than B.S. degree work- 
ing under supervising professional teacher? identify 
competencies needed by aides? identify competencies 
teachers need to work with aides. 



One group suggested the concepts of: 1) teacher competencies as 

a team teacher, 2) paraprofessional as well as classroom specialist- 
generalists needed to be involved in the problem area statement. 

This group also suggested this problem area should include "spe- 
cific sensitizing methods to promote empathy for needs of those 
different from themselves." One group thought this item should 
be omitted because evidence is lacking in establishing a differ- 
ence in effectiveness of strategies. 



Number 6 (p. 139) on the Tentative List had three written 
reactions. The original statement of the problem area follows. 

6. Test effectiveness of various methods of sensitiz- 
ing home economics teachers to the needs, goals, 
aspirations, practices, and attitudes of persons 
different from themselves. 

One group wanted it stated as "ways of understanding people dif- 
ferent from oneself." Another group agreed with it as it was 
stated. Another group thought this item should be omitted because 
evidence is lacking in establishing a difference in effectiveness 
of strategies. 



The remaining six priority problem areas on the Tentative 
List were ranked second in priority by the experts using the 
DELPHI Technique. The groups either did not react to these items, 
or they wished to react to the entire list of priority problem 
areas, or they wanted to create their own list of priority prob- 
lems. 



Three groups endorsed problem area number 50 (p. 117) from 
Questionnaire Number 2. The original statement of the problem 
area follows. 
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50. Using something like the Flanagan Critical Inci- 
dent Technique, determine critical areas of deci- 
sion-making in consumer behavior. What sorts of 
things cause significant successes and failures 
in the activities of consumers? In middle socio- 
economic homes, in lower socioeconomic homes, in 
rural and urban ghetto homes? 

Two groups endorsed priority problem area number 21 (p. 113) . 

Other priority problem areas being endorsed by at least one group 
were number 20 (p. 113) if combined with numbers 55, 30, 31, and 

40 <pp. 115-117) . 

At least two groups wrote that a priority problem was evalu- 
ation of curriculums already developed (by demonstration, by test- 
ing) to see what is effective where. A group suggested a priority 
problem as, "ways to get coordinated effort and maximum use of 
resources from other people in the schools as well as community 
agencies, businesses, etc." One group specified we need research 
relating to the "most effective in-service education in order to 
effect change." 

One group structured a different approach to use in arriving 
at priority research problems. This approach outline follows. 

Research problems need to follow a sequential development of 
priorities. The following recommendations may not be mutually 
exclusive nor are they intended to be limiting. 

Recommendations of criteria to develop research model: 

1. Develop conceptual structure of consumer and homemaking 
education. 

2. Teacher education and content specialists work together 
to delineate the content. 

3. Establish curricular models for various school systems. 

4. Develop assessment instruments. 

5. Explore curricular adaptations for various socioeconomic 
groups . 

6. Explore process of increasing change procedure. 

Suggestions for implementing the above criteria were: 

1. Establish a conceptual structure of research in Consumer 
and Homemaking Education. 
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2. Select research components within the structure by vari- 
ous institutions and agencies according to own interests 
and abilities. 

3. Select research areas from any point on the continuum. 

4. One central group, such as The Center for Vocational and 
Technical Education, serve as a disseminating agent for 
all aspects of the program. 

Conflicting Ideas Summary . One central conflict was trying 
to establish the interrelationship of the concepts of "consumer 
and homemaking education" and "home economics education." Ideas 
on what constitutes defensible research on Consumer and Homemaking 
Education were not shared in common. 



SUMMARY OF SELECTED SMALL GROUP MEETINGS 

One member from each of the eight groups met at the close of 
the afternoon session. Their task was to synthesize the group re- 
ports into a list of priority research problems on Consumer and 
Homemaking Education. Their recommendation to the Coordinator at 
the conclusion of their meeting was to adjust the next day's pro- 
gram to include time for the same groups to meet again and using 
Questionnaire Number 2 (pp. 111-119), reorganize the priority re- 
search problem areas into broad categories, eliminating or adding 
any research problems in the process. 

See Appendix E (pp. 153-158) for the final revision of the 
group deliberations. 

Participants interested in research focusing on a specific 
category met together and discussed feasible studies. The specific 
categories were: curriculum development, occupational related, 

and teacher education. Appendix F has the names of participants 
and their research interest area. 



SELECTION OF RESEARCH 
PROBLEM AREAS AND PROJECTS 

Participants were invited to the conference because of their 
involvement in Consumer and Homemaking Education activities. They 
also were invited because of the type of expertise and/or position 
they have (leadership, administration, and/or research). Within 
this framework, discussion as to allocation of resources needed 
for conducting research projects and considerations in the selec- 
tion of research projects would be most purposeful. Even though 
it was realized by the conference planners that a firm commitment 
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of resources for specific research projects could not be made at 
the conference, most of the considerations necessary in making 
these decisions would be presented at the conference and indica- 
tions of interest to the priority research problem areas could be 
noted . 

A panel responded to the topic of "Decision-Making: Con- 

siderations in Allocation of Resources and Selection of Research 
Projects." Summary papers of the three presentations follow. 
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NATIONAL RESEARCH CONFERENCE ON 
CONSUMER AND HOMEMAKING EDUCATION 

June 3, 1970 



RUTH HO VERM ALE 



I am representing a state-supported institution, serving a 
land-grant function for resident instruction and research in home 
economics with a strong liberal arts orientation. What I have to 
say works in my institution; the ideas should be applicable to 
other situations. An administrator needs to consider three key 
areas in allocating resources for research; they are: 1) per- 

sonnel, 2) budget, and 3) communication. 



PERSONNEL 

1. Differential staffing — persons with less than bachelor's 
degrees; undergraduate students; part-time personnel; 
state department employees — secondary teachers; examine 
all personnel to extend the scarce resources we now have. 

2. Released time — review responsibilities and see what can 
be delegated to other individuals — even the administrator. 
The administrator may need to take on additional respon- 
sibilities . 

3. Working environment — researchers need a place to work — 
ideally space needs to be allocated for research; if 
space is not available the immediate environment can be 
made to be more conducive to research. 

4. Cooperation between institutions provides stimulation for 
researchers and makes it possible to broaden the base of 
research. 

5. Cooperation within institutions — home economics education 
and other subject-matter areas within home economics; 
home economics and other disciplines on campus; research- 
able problems need the cooperative efforts of all person- 
nel. The problems you have been discussing at this con- 
ference are concerns of all home economists not just home 
economics education. 

6. Graduate students — cooperative research between institu- 
tions; when are we going to have graduate students stop 
doing "bits and pieces" of research when a part of the 
whole would be more meaningful? 
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7. Research Coordinating Units — need to determine what is 
appropriate for RCU and local institutions. 

BUDGET 

1. Such money is available on many campuses to begin re- 
search — is money available to home economists as well as 
faculty from other disciplines? 

2. Federal funds are allocated to each state — do you know 
what is being allocated to your state? 

3. Need to have projects "in your hip pocket"--if money is 
available, project is ready to move. Funds may not be- 
come available until the end of FY. Initial grants used 
for setting up project; implementation comes later. 

Once funds are allocated to an institution, it may be 
easier to get additional funds. 

4. State Department of Education may be able to provide 
support for a part-time research assistant and supply 
funds for summer employment. 

5. State appropriation for research may have relatively few 
"strings attached." 



COMMUNICATION 

1. Every administrator needs to be a politician — needs to 
know the way the "wind is blowirg"; needs to be aware 
of the educational structure within your state. 

2. Administrator needs to keep abreast of what is being done 
locally, regionally, nationally; needs to exchange ideas 
with all concerned. 

3. The Association of Administrators of Home Economics 
(land-grant colleges and state universities) has under- 
written the cost of a National Long-Range Projection for 
Research in Home Economics. The study was made to iden- 
tify research areas of greatest import, project personnel 
needed and examine ways of maximizing personnel potential. 
The printed report of the study will be available this 
fall. Each member institution will receive copies for 
distribution. 

4. There is a need to bridge the communication gap between 
the high school teacher and the college instructor. May 
I urge you to make plans for each faculty member who 
teaches your majors to visit a public school this fall. 
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NATIONAL RESEARCH CONFERENCE ON 
CONSUMER AND HOMEMAKING EDUCATION 

June 3, 1970 



HAROLD BINKLEY 



We have a Department of Vocational Education, in the College 
of Education, at the University of Kentucky. The Department in- 
cludes preservice and in-service teacher education in the service 
areas of agriculture, business, distribution, home economics, and 
trade and industrial education. We also have an Instructional 
Materials Laboratory with nine specialists and the Research Co- 
ordinating Unit with five professional staff members. 



AREAS FOR DISCUSSION 

— How faculty loads vary for those doing research. 

— How budgets are planned to include graduate assistants. 

— How contracts are provided for state-needed research 
projects. 

— How travel budgets are included for researchers. 

— How assistance is given in preparing research proposals. 



TEACHER EDUCATION 

The teaching load of staff members may vary depending on the 
kind and amount of research to be done. Faculty members in the 
"Regular Series" may program and devote one-third to one-half of 
their time to research. Faculty appointed to "Special Title 
Series — Student Teaching" are not required to do research; however 
many do. 



Faculty load may vary depending on the Service Area due to 
the State Department's desire for certain amount of effort to be 
devoted to research. For years we had the understanding that the 
equivalent of one-half time of a qualified staff member would be 
devoted to research jointly agreed upon between the State Office 
(State Supervisor) , and the home economics staff at the University. 
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BUDGETS FOR GRADUATE ASSISTANTS 



What budget? 



Teacher education 
RCU 

Special research projects 



2 or 3 assistants 
4 to 6 assistants 
1 or more 



CONTRACTS FOR STATE-NEEDED RESEARCH PROJECTS 

Two types hive good possibilities here. 

1. Direct contracts between the State Department of Educa- 
tion and the Department of Home Economics Education in 
which the Department of Home Economics Education agrees 
to do a research project of a specified scope, in a set 
time, at a set cost. Money is transferred to the Uni- 
versity account number or the State Department agrees 
to support the Department of Home Economics Education 
for an additional given number of dollars. 

2. Contract for research with and thru the State RCU. Pro- 
posals are drawn up for specific research proposals with 
budget attached. The proposal is submitted to the State 
RCU which evaluates the proposal and recommends approval 
to the State Director of Vocational Education. For the 
last three years, the Bureau of Vocational Education, 
State Department of Education has set aside $40,000 
against which special research proposals have been sub- 
mitted. 



We have been able to secure research equipment for the RCU 
through joint purchase by the University and the Bureau of Voca- 
tional Education. Large research projects have included purchase 
of certain equipment in budgets. 

Vocational personnel have available for their use all of the 
equipment of the research and statistical laboratory of the Col- 
lege, located in the same building. 



RESEARCH NEEDED EQUIPMENT 



TRAVEL BUDGETS 



— Teacher education budgets. 

— Research Coordinating Unit budgets. 



Budgets for special research projects include the travel 
(both in and out of state) necessary or desirable for the 
researchers to attend pertinent research conferences and 
activities . 



ASSISTANCE PROVIDED RESEARCHERS 

The staff in the RCU provides assistance to any and all re- 
searchers whether teacher education, state staff, individual re- 
searchers in. the design and management of research proposals and 
projects whether funded through the RCU or otherwise. This staff 
provides assistance to any individual or agency in the state de- 
siring to do research related to Vocational Education. Their 
assistance can and does include assistance from the conception, 
design, and proposal writing through the writing of the final re- 
port and dissemination. 



ALLOCATION OF RESOURCES 

A State Advisory Committee of the State Research Coordination 
Unit evaluates proposals for research and development projects and 
recommends their approval or disapproval to the State Director of 
Vocational Education. 

— What findings from research can make the greatest dif- 
ference in the lives of needy individuals? (Use of illus- 
tration from agriculture in the difference of corn yield.) 

— Cost-benefit analysis. 

Decide who is going to do "what" and not everyone can select 
her "thing" to do. 

Involve people in other areas of Vocational Education. 
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NATIONAL RESEARCH CONFERENCE ON 
CONSUMER AND HOMEMAKING EDUCATION 

June 3, 1970 



CLAYTON OMWIG 

AREAS FOR DISCUSSION 

1. Some background on Research Coordinating Units (RCU's) . 

2. How a total State system can go about setting priorities. 

3. The assistance an RCU can be to the researcher and con- 
sumer . 

4. Funding outlook. 

RESEARCH COORDINATING UNITS 

WHAT IS AN RCU? 

a. Research and Development arm of State program. 

b. To facilitate desirable changes in vocational educa- 
tion . 

c. Predicated on involvement of all. 

d. Staff — human resources, educational resources, man- 
power, training, and information services. 

e. Next year — planning, evaluation, exemplary program, 
information services, training. 

LEGISLATIVE MANDATE 

a. Vocational Act of 1963 — direct funding from Bureau of 
Research. RCU not mentioned in 1965. 

b. Vocational Amendments, 1968 — specifically mentioned 
RCU's. Seventy-five percent through department funds. 

STIMULATE RESEARCH 

a. With and among agencies and universities, etc. 
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b. Difficult — not too much interest exists (faculty 
loads, etc.). 



COORDINATE 

-- Avoid unnecessary duplication. 

— Formulate long-range and annual research plans. Must 
consider: 

a. Information for decision-makers — people, jobs, train- 
ing. 

b. Program development and improvement. 



SETTING PRIORITIES 

If adequate funds are available, then we can do many things. 

If funds are limited, we must look at internal needs of State. 

HOW? 

a. State Plan priorities. 

b. 68 Amendments priorities. 

c. Priorities as seen by administrators. 

d. Priorities as seen by division heads and institutions 
of higher education. 

e. Priorities as seen by State Research Advisory Council. 

f. Priorities as seen by coordinating committees. 

g. Priorities as seen by RCU staff. 

h. Priorities as seen by Washington (USOE) . 

USOE has set of five. 

1. Reading. 

2. Early childhood education. 

Educational systems and how to best use financial 
and manpower resources. 



3 . 



4. Basic research (10 percent). 

5. Vocational education. 

Vocational Education priorities from Washington. 

1. Program and administration--considerations to 
make vocational education more effective. 

2. Learning environments. 

3. Exemplary instructional systems. 



WHAT? 

1. Interested in quick payoff. 

a. Some good research not carried to practitioners. See 
what is good — add additional investment — get it to 
where the practitioner can use it. 

b. Combine pieces of research. Look at related re- 
search — again add additional money, if needed, get 
to the practitioners . 

c. Look at good programs that exist in the military, 
business and industry, other state programs, special 
programs, etc. Pick up what is good and demonstrate 
the materials or techniques. 

2. Programmatic research — where we (RCU) place the priority. 
Let the Centers (Ohio and North Carolina) do most of the 
pure or basic, plus some university people. 

3. Demonstration and innovative programs. Effect of seeing 
it in action (in own state) . RCU furnish seed money — 
can pay minimum amounts of "extra costs." 



RCU ASSISTANCE 

Provide assistance through some funding and microfiche for 
literature reviews. 

PROJECT DEVELOPMENT 

Be very selective. 

a. Funds are limited. 
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b. See what will have the greatest impact on the program. 



c. See what is of urgency to the State program. 

1. Evaluation. 

2. Follow-up. 

3. Information for planning. 
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PART II 

CONFERENCE OBJECTIVE 3: 



To describe research strategies, methodologies, and 
institutional-state resources to contribute to 
the solution of critical problem areas. 



PART II 



PRE-CONFERENCE ACTIVITIES 

A visit to The Center for Vocational and Technical Education 
was arranged by the planners. Specialists in Research and Devel- 
opment and Information were contacted prior to the conference. 

They were asked to explain a project/ or an idea, or a system 
which they were responsible for implementing. The criteria for 
selection were: 1) its use in the conference as a part of the 

design; and 2) the generality of its application for leadership, 
administrative, and research personnel. 

A Research and Development Specialist at The Center for Voca- 
tional and Technical Education took one of the research priority 
problem areas and stated this problem in researchable terms by 
delineating the variables. He then selected a possible research 
study, which when completed, might have contributions to make 
toward the partial solution of the priority problem. Suggested 
project methodology, statistical design, and possible outcomes 
were organized by the specialist. 

CONFERENCE ACTIVITIES 

THE CENTER FOR VOCATIONAL 
AND TECHNICAL EDUCATION 

The facility which houses the administrative staff, research 
and development staff, and the ERIC Clearinghouse for Vocational 
and Technical Education was seen by the conference participants. 
Special attention was given to the library which serves the staff 
members, the video tape recording studio, and the retrieval system 
for vocational education information. 

Miller presented a paper titled "The Challenge' of Being One." 
This presentation can be found on pages 29-31. 



ERIC AS A BIBLIOGRAPHICAL RESOURCE 

The Bibliography of Research on Consumer and Homemaking 
Education, which was prepared for the conference participants, 
used as its source the ERIC system. Explanation of the system 
was made to the participants and two microfiche from the Bibliog- 
raphy were presented to each participant. 
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The basic information presented by Magisos is contained in 
the Bibliography , Bibliography Series No. 6, VT 010 860, ERIC 
Clearinghouse on Vocational and Technical Education, The Center 
for Vocational and Technical Education, The Ohio State University. 



CIPP MODEL FOR EVALUATION 

The concepts involved in the CIPP Model are being applied in 
the management of Center activities and research projects. Some 
of the concepts were also involved in the planning for and evalua- 
tion of the conference. An explanation of the b,asic concepts and 
some examples of implementation of the ideas were presented to 
the group. The paper presented by Walker is on pages 32-38. 



SIMULATION PACKAGES IN LEADERSHIP DEVELOPMENT 

A research project in the State Leadership Task Force has as 
an objective the development of simulated materials to aid state 
leadership personnel in decision-making. The group received in- 
formation related to the effectiveness of simulated materials and 
the present availability of simulated leadership material. The 
paper presented by Ward is on pages 39-49. 



MICRO-SUPERVISION 

A project, completed at The Center, tested the feasibility 
of partial training of vocational-technical teachers as super- 
visors and teacher educators. Learning how to conduct a conference 
with beginning teachers was achieved by use of a micro-supervision 
technique. Explanation was given of the technique, the critique 
forms used for evaluation, and plans for dissemination of the idea. 
Cotrell and Chase made this presentation and an overview of the 
report of the study can be found in the Third Annual National Vo- 
cational-Technical Teacher Education Proceedings Micro-Teaching 
and Video Recording , Leadership Series No. 25, VT 010 228, The 
Center for Vocational and Technical Education, The Ohio State 
University. A more detailed research report is in the process of 
being published. 



DELPHI TECHNIQUE 

The tentative list of research priority problem areas in Con- 
sumer and Homemaking Education was developed for the conference by 
use of the DELPHI Technique . Further explanation and an example 
of using the technique in a dissertation study were given to the 
group by Gray and is included on pages 50-57 . 
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"the challenge of being one" 

June 3, 1970 
4. J. MILLER 



My role this afternoon is to try to challenge you to be "one." 
Now, this challenge might be interpreted in several different ways. 
For example, if the word "one" were interpreted as indicating pre- 
eminence, we might talk about the Ohio State football team. Each 
year Woody Hayes challenges the team to be one — number one. Some- 
times they make it, but most of the time they don't. 

If the word "one" were used to refer to an individual unit or 
thing, we might look at organizational examples of Vocational Edu- 
cation some years back. Then, most everyone knew that each tradi- 
tional service area was a unique and mutually exclusive entity 
which dealt with problems mostly unique to that service area. 

Being "one," in that sense of the word, is not my challenge for 
you today . 

The "one" that I wish to define this afternoon is the "one" 
that denotes a sameness in kind and quality; a oneness that might 
define a group of vocational education researchers, regardless of 
their vocational background or occupational experience, who are 
interested in a common educational research problem affecting a 
clientele for whom they have a mutual concern. 

This concern for "oneness" is only one of a number of the 
concerns for vocational and technical education expressed by both 
the 88th and 90th Congress. 

In 1963, the 88th Congress gave attention to some of the 
fundamental problems of vocational education for the first time 
since 1917 . They were motivated by high unemployment among the 
youth of our country. Another factor motivating Congress' concern 
was the importance of formal preparation for employment in an in- 
creasingly technical and sophisticated economy. The Vocational 
Education Act of 1963 addressed itself to these challenges. 

The authors of the Vocational Education Act of 1963 recognized 
the need for flexibility in order to meet the demands of changing 
society and institutions. Therefore, the act included an evalua- 
tion system. This evaluation system required the appointment of 
a National Vocational Education Advisory Council to appraise the 
results of the act each five years and to recommend administrative 
and legislative improvements. In 1966, the first National Voca- 
tional Education Advisory Council was appointed. Based upon the 
report of the National Advisory Council, the original Vocational 
Education Act of 1963 was amended and updated. 
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The Vocational Education Act of 1963 introduced two new basic 
purposes in Vocational Education: first. Vocational Education was 

to serve the occupational needs of all the people in the community 
through unified programs rather than to train people in separate 
programs of selected occupational categories. Secondly, a new 
group was to be served: the persons who could not succeed in a 

regular vocational education program because of educational, socio- 
economic, and other obstacles. So far, there is little evidence 
that either of these major purposes has been accomplished to any 
significant degree. 

If we look at the various recommendations of the National 
Advisory Council and try to synthesize these recommendations into 
one broad encompassing guide for Vocational Education, it is clear 
that this guide must be " the development of a unified system of 
Vocational Education for the United States ." That is, we must 
strive to achieve a ''oneness" of purpose in meeting the training 
needs of people. 

Let us look for a moment at some of the characteristics of a 
unified system of Vocational Education that should be achieved 
through "oneness" of purpose. 

1. Occupational preparation should begin in the elementary 
schools with a realistic picture of the world of work. 

Its fundamental purposes would be to familiarize the 
student with his world and to provide him with the in- 
tellectual tools and rational habits of thought to play 
a satisfying role in it, 

2. In junior high school, economic orientation and occupa- 
tional preparation should reach a more sophisticated 
stage with study by all students of the economic and 
industrial system by which goods and services are pro- 
duced and distributed. The objectives would be exposure 
to the full range of occupational choices which would be 
available at a later point and full knowledge of the 
relative advantages and requirements of each. 

3. Occupational preparation should become more specific in 
the high school, though preparation should not be limited 
to a specific occupation. Instruction should not be 
overly narrow, but be built around families of occupations 
or industries which promise expanding opportunities. All 
students outside the college prep curriculum should ac- 
quire an entry-level job skill, but they should also be 
prepared for post-high school vocational and technical 
education. Care should be taken that pursuit of a voca- 
tionally oriented curriculum in high school does not 
block the upward progress of the competent student who 
later decides to pursue a college degree. 
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4 . Vocational preparation should be used to make general 
education concrete and understandable. Curriculum mate- 
rial should be prepared for both general and vocational 
education to emphasize these relationships. 

5. Some formal post-secondary occupational preparation for 
all should be a goal for the near future. Fourteen years 
of free public education with a terminal occupational 
emphasis should be a current goal. 

6. After initial preparation for employment, many will want 
to bolster an upward occupational climb with part-time 
and/or full-time courses in programs as adults . These 
should be available as part of the regular public school 
system. 

7. Occupational preparation need not, and should not, be 
limited to the classroom, the school shop, or to the 
laboratory. Greater emphasis must be given to on-the- 
job training and utilization of training situations in 
business and industry. 

8. The school must work with employers to build a bridge 
between school and work. The school must accept the 
responsibility of student placement and follow-up. 

9. A full range of remedial programs must be provided by the 
public school system to meet the needs of all students 
who have problems in succeeding in regular vocational 
programs . 

Part F of the 1968 Amendments addresses one specific area of 
Vocational Education — that is, problems which relate to the needs 
of people concerning preparation for the roles of a homemaker, a 
homemaker -wage earner, and as a consumer, with special attention 
given to the socially and culturally deprived. 

The education and training needs of the wage-earner and the 
consumer transcend traditional vocational service areas as do the 
special educational needs of the socially and culturally deprived. 
The challenge of being "one" is the challenge given to this group 
to address the problems from the professional vantage point of one 
group — educational experts — dedicated to the identification of 
appropriate research problems for systematic solution. 

It is indeed heartening to welcome you to The Center this 
afternoon, a group of national leaders from a variety of voca- 
tional service areas and substantive backgrounds. The diversity 
of your vocational backgrounds, administrative positions, and pro- 
fessional expertise are adding a unique vigor to this research 
planning conference. 
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"what's WRONG (RIGHT) WITH EVALUATION?" 

June 3, 1970 

JERRY WALKER 



Allow me to make some positive points about evaluation by 
trying to present and then react to some of the negative things 
which have contributed to evaluation's being as ineffective as 
we find it in American education today. 

First, I will try to point out some of the more conspicuous 
barriers to effective evaluation, then attempt a definition and 
last, try to convert some of the initial negative remarks into a 
more positive vein by suggesting means of circumventing some of 
the barriers. 

What then, are some of these barriers to effective evalua- 
tion? 



EVALUATION IS THREATENING 

It is threatening because, if effective, it necessarily em- 
bodies the concept of accountability . Among the various defini- 
tions of evaluation and among the various functions it is to per- 
form, one clea.: premise which emerges is that evaluation is a 
means for assessing the extent to which any education program, 
project or activity meets its own specifications (internal valid- 
ity) . To determine if specifications are met, they first must be 
stated; they are stated in the form of demonstrable objectives. 

In spite of the lip service to the contrary, we really don't like 
to set forth, in measurable terms, the outcomes of our intentions 
and efforts. Yet, good evaluation forces us to and it lets us 
know if we have not made it. 



EVALUATION VERSUS RESEARCH 

In particular, the relationEship between evaluation and re- 
search is consistently misunderstood. They differ, I think, pri- 
marily in the extent to which evaluation is targeted to the in- 
formation needs of specified decision-makers . The activities 
(i.e., evaluation and research) may be identical, the purposes 
differ. Excellent research is often very poor evaluation. Good 
evaluation may be very poor reseiarch (in the case of decision- 
makers who do not have the time to be very choosy about the 
quality of information they need) . 
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DECISION-MAKERS ARE BETTER REACTORS 
THAN THEY ARE ACTORS (PLANNERS) 



It is the unusual decision-maker who can specify his informa- 
tion needs completely and accurately in advance or who can speci- 
fy the type, priorities and weights of the criteria which he will 
employ in making his decision. Most, however, are able to tell 
the evaluator, post facto , what they did not want and what was 
missing in terms of the information provided by the evaluator. 
Thus, the evaluator is forced to pursue a "shotgun" approach 
where he blasts all the information that he can obtain from any 
and all sources in the general direction of a decision-maker (if 
he is fortunate enough to know who the decision-makers arc) in 
the hopes that some of the information will meet some of the re- 
quirements and criteria of some of the decision-makers some of 
the time. It is not very effective or efficient; it is necessary. 



IT SERVES MULTI- AND OFTEN CONTRADICTORY PURPOSES 



For example, a single evaluation effort may be perceived by: 
Role Pur pose 

Project Director Superfluous, platitudinous window 

dressing to appease funding sources. 



Project Functionary An honest assessment and critique 

of the relative strengths and 
weaknesses of the project, the 
reasons for either, and the rec- 
ommendations for improvement. 



Evaluator A careful and conscientious re- 

porting of the information re- 
quired by pertinent decision- 
makers . 



Funding Sources 



Clients (Students/ 
Parents) 



Answers to the Questions; 

Is this project making a dif- 
ference? 

Is it accomplishing what it set 
out to do? 

Another extravagant and unimportant 
waste of tax dollars. 



EVALUATION IS EXPENSIVE 

It is expensive in terms of time, monetary and personnel re- 
sources. People do not fully appreciate' the precious commodity 
which is information. For some reason we tend to think that it 



is in abundance, that it is at our finger tips and that to obtain 
it requires little planning and even fewer resources. Quantita- 
tively, this is probably right; information is in abundance, it 
is literally at our finger tips and it is inexpensive. But, if 
the information is to be pertinent , then it becomes very precious 
and very expensive in direct proportion to the specificity and 
uniqueness of the decision-makers' needs and criteria. 

EVALUATION AS AN EVENT RATHER THAN A PROCESS 

Evaluation is often considered as simply an additional step 
in the life of an educational program or project — usually a last 
step. This is true for two reasons I think. One is the limited 
definition of evaluation as the act of making a judgment or se- 
lecting among sets of alternatives. To define evaluation as (just) 
decision-making ignores the process of obtaining the information 
on which decisions are based. The other reason for ignoring the 
continuous process which must characterize good evaluation is the 
traditional (but unnecessary) emphasis placed on "end of project" 
outcomes. This emphasis is placed because of the traditions of 
experimental design demanding pretesting and post-testing, because 
of the nature of school years or semesters which "end" and then 
start again (as if learning did) , because of the fiscal year cycles 
which also end and renew, and because of the artificial division 
of labor between project functionaries and evaluators — a division 
of labor which causes project personnel to say in effect, "We're 
through now; let's call in the evaluators. 

These are but a few of the reasons for ineffective evalua- 
tion; many other reasons exist as generalizations and countless 
reasons could be documented where evaluation has failed in specif- 
ic instances and circumstances. But even the few reasons cited 
here should be sufficient to support the growing belief that the 
theory and practices of educational evaluation are in need of im- 
provement through changes which are at least as widespread and 
pronounced as evaluation's ineffectiveness. To me, this means 
that truly radical change is necessary. 

Now in the brief space available here, we cannot solve the 
problems besetting evaluation.! We can't even list them. We can, 
however, try to turn the few negative points that have been made 
into positive statements — statements which capture the challenge 
and potential which sound evaluation holds for us as evaluators, 
educators, administrators, and most importantly, the challenge 
and potential it holds for increasing the range of opportunities 
for the children to and for whom we are all responsible. 



ISpace, as a matter of fact. Is not the reason; It's simply 
a convenient excuse. The reason, as we'll discuss, is bound up 
in our ignorance of what the stuff we call evaluation Is, ought 
to and can be. 
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A DEFINITION 



Let's first attempt to define evaluation — a prescriptive or 
"ought to be" definition rather than the dismal descriptive one 
we've already outlined. Then, looking at each of the reasons 
cited for ineffective evaluation, perhaps a few strategies to 
solve or circumvent these barriers can be suggested. 

By paraphrasing Daniel Stufflebeam, evaluation can be de- 
fined as the process of obtaining and providing useful information 
for making educational decisions. 2 This definition is essentially 
consistent with those offered by Phi Delta Kappa, The UCLA Center 
for the Study of Evaluation, Robert Stake, and many others. There 
is little need to parrot the several slightly different definitions 
of evaluation and there is little reason to quarrel with Stuffle- 
beam's because it includes the three key concepts which I think 
must characterize evaluation, namely, that (1) useful information 
is obtained and provided for (2) educational decision-makers 
through the (3) continuous evaluation process . Keeping this 
definition in mind, we can return to the barriers to evaluation 
which were discussed earlier. 

Threatening ; If the decision-makers within an educational 
program would consider evaluation primarily as a tool which will 
serve their ends and not those of an amorphous "they" out there 
somewhere, it would not be threatening. And the amorphous "theys" 
(often holders of federal, state and local purse strings) should 
see evaluation as an integral part of any educational program 
instead of a means for adding mass and additional spikes to the 
"big sticks" which often represent their philosophy of motivation. 
Evaluation ought to provide information for planning decisions 
(What should be done?); structuring decisions (How to do it?)? 
implementing decisions (Are we doing it?) ; and recycling decisions 
(Did we do it?) . Most of these decisions are internal to an edu- 
cational program and if the program is to be successful (a nec- 
essary but not sufficient condition for which is continued fund- 
ing) , then the decisions must be sound and they must be rational 
(i.e. , must be made on the basis of timely and pertinent informa- 
tion) . Nothing threatening should come from an evaluation system 
which provides this information. No business or industry could 
survive without an information system to serve these kinds of 
decisions, yet educational managers (who deal in products of 
infinitely more worth than any business) resist such a system. 

Evaluation and Research ; There is no simple answer to the 
long and often heated debate between researchers and evaluators 
but if a basic understanding could be reached and agreed upon, 
the confusion would lessen; the cooperation would increase. I 



‘D i rector , 



Eva I uat i on Center , 



The Ohio State University. 
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think the principal lesson that evaluators could learn from re- 
search is the need and tools for obtaining quality information 
(e.g., objective, replicable, valid, reliable, etc.). The 
educational researcher, on the other hand, should come to the 
realization that the production of knowledge is a means and that 
to be concerned with "knowledge for what (or whom) is not a 
prostitutionalization of self or discipline. 3 

Reactive Decision-Makers ; In the first place, this "charge" 
is not unique to decision-makers. For all of us, our hindsight 
is a lot clearer than our foresight. But the fact that decision- 
makers are not exempt from this human frailty is critical to the 
evaluator who, by definition, must provide information meeting 
the needs and criteria of the decision-maker. One possible strat- 
egy to reduce (not solve) this problem is to send "trial balloons" 
up before substantial resources are committed to the collection of 
information. A useful device might be a topic or sentence outline 
of the final report which the decision-maker will ultimately re- 
ceive. A list of questions to be answered could be reviewed by 
the decision-makers. Several meetings between evaluator and 
decision-makers with successive iterations of the information 
requirements should be conducted. 

Obviously, the evaluator has the responsibility to tactfully 
but persistently prod the decision-maker to make him aware of the 
criticality of specifying his information needs as clearly and 
completely as possible. Another approach, already mentioned 
briefly, is the "shotgun" approach which following the rule of 
thumb "When in doubt (about what information you want), ask it." 

We see this manifest in voluminous questionnaires and other in- 
struments and subsequently in the evaluation reports. The trouble 
with the shotgun approach is that since resources are always 
limited, the quality of the information is sacrificed for quantity 
(e.g., "and now a chapter on the anecdotal comments of the con- 
ferees. . . ! " ). Also, it is nearly axiomatic that the length of 
a report is inversely proportionate to the probabilities of it 
ever being read. 

EVALUATION FOR MULTI-PURPOSE, AS A PROCESS, 

AND AS BEING EXPENSIVE 

I think that a few summary remarks about evaluation should 
address themselves to the three additional examples of barriers 
to ef fec-'i.-e evaluation: that it is perceived (1) as serving 

multi- contradictory purposes, (2) as a discrete event, and 
(3) as .-^ing expensive. It would be foolish tc defend evaluation 
from these charges because they are glaringly true. The first 



^RE: Theme of 1970 American Education Research Association 

Annual Convention: "Knowledge for What?" 
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two are true because of our confusion about what evaluation is 
and how it is to be used. And we are confused because of the 
historic framework from which modern evaluation theory has 
emerged — a history which has seen evaluation equated with psycho- 
metric measurement, with assessment, with judgment of any sort, 
with descriptions of outcomes, and with {just) decision-making. 

As for its expense , one can only counter with the question 
as to worth of knowing what needs to be done, how to do it, if it 
is being done and whether it was accomplished. We cannot safely 
or effectively walk across the street or rocket to the moon with- 
out this knowledge; how can we educate the nation’s children with 
out it? 




37 





38 



SIMULATION MATERIALS AND TECHNIQUES: 
INSTRUCTIONAL TOOLS FOR VOCATIONAL 
EDUCATION LEADERSHIP PREPARATION 

June 3, 1970 



DARRELL L . WARD 



Leaders of vocational education today have a tough job! Not 
to take anything away from our past leaders and their accomplish- 
ments, but — they had it good when compared with the complex prob- 
lems besetting vocational education leadership today. Never has 
vocational education been called upon to do so much with — com- 
paratively — so little. Today's leaders are called to provide, in 
fact demanded, to execute leadership for poorly defined, little 
understood, and highly desired accomplishments. A Herculean ef- 
fort will be required. The job cannot be done with our present 
capabilities, resources and postures of action. New methods, in- 
creased efficiency, and greater effectiveness will be required. 



Many inputs will be required to accomplish the tasks ahead. 

As in the past, many of these inputs will be conceived, borne and 
reared by the innovative local leader who "gets the job done down 
where the rubber hits the road." However, state and national 
leadership must and is moving to provide assistance, training, 
and resources commensurate with the job to be done. Legislative 
action, the funding and implementation of the Educational Profes- 
sional Development Act's vocational education fellowships, is one 
example. The programmatic and continuing effort of The Center 
for Research and Leadership Development in Vocational and Technical 
Education, The Ohio State University, to produce materials and 
provide training for Vocational Education leaders is another ex- 
ample of help at hand. Progress being made by other R and D in- 
stitutions and operating agencies shows bright promise on the 
horizon. Certainly, the activity at this meeting is indicative 
of the need, desire, and importance of increased and improved 
preparation for present and potential vocational education leader- 
ship personnel. 



Training materials which utilize simulation techniques to 
prepare people to use new leadership tools and for teaching lead- 
ership skills are emerging as one vital and meaningful hope in our 
arsenal of resources with which the problem may be attacked. The 
area of simulation training is not new. But, judging by the recent 
burst of activity in this field and the proliferation of publica- 
tions lauding its potential and presenting its methods, it has 
recently been given new life. 



» 



Many definitions of simulation exist. They are almost as 
numerous as are writers on the subject. Seemingly the area is 
becoming very popular. A definition that I find acceptable has 
been presented by Weem in an article titled "Simulation as a 
Method of Instruction in Science Education." His definition 
states--" simulation is an imitation of real circumstances aimed 
at providing a learning environment." Weem further states that, 
"simulation is a technique by which the essential features of 
some object or process are abstracted and recombined in a model 
which represents the original and can be manipulated for the 
purpose of study or instruction. " I Perhaps the most often quoted 
definition of simulation is that of Professor Harold Geutzkow 
of Northwestern University. His definition is that "simulation 
is an operating representation of central features of reality. "2 

In spite of how you or I or others might define simulation, 
simulations are useful teaching tools. The quality that distin- 
guishes them from other pedagogical techniques, that gives them 
their particular power, may not be immediately apparent. Simula- 
tions are often the only means available to duplicate processes 
in a neutral context. This, I think, has tremendous meaning for 
us in education and in particular vocational education as we move 
into comprehensive vocational education planning and cost account- 
ability for our programs of occupational preparation. An eclectic 
definition which I would like to propose is this. Simulation is 
an operating representation of the central features of a real 
circumstance aimed at providing the learner with a relatively 
safe, simplified, and germane learning environment (see Figure 1) . 
Too often the real world about which we must learn is too remote, 
too complex, or too unsafe to be effectively utilized as a teach- 
ing and learning situation. If the real world does not exhibit 
these characteristics to a prohibitive degree, it is often the 
best place to learn, i.e., our successful on-farm projects in 
vocational agriculture, cooperative work experience in distribu- 
tive education, and the on-the-job training of many vocational 
education programs and complimentary educational endeavors . 

However, when the characteristics, remote, dangerous, and/or 
complex, are present, simulations can draw from the real world 
the central features of an actual circumstance, place them in an 
operating representation or model and thus provide the learner a 
relatively safe, simplified, and relevant learning environment. 
Please note — a simulation is nothing without the learner. 



^Richard D. Weem, "Simulation as a Method of Instruction, 
Science Education . 

2 

H. Geutzkow, ed . , Simulation in International Relations: 
Developments for Research and Teaching, Englewood Cliffs, New 
Jersey: Prentice Hall, 1963, p. 25. 
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FIGURE I. EDUCATIONAL SIMULATION DEFINED 



Simulation merely provides a learning environment. Placing the 
learner in the environment along with a high degree of involve- 
ment in an action program is the job of both the teacher and the 
learner himself. 

To further illustrate what seem to be valid reasons for 
using simulations in preference to some other teaching method, I 
would like to provide some examples of where simulations have been 
used or have particularly great potential. Actual war is too 
dangerous, and too late to put the learner into it for basic and 
combat training. Thus the military has long used war games to 
prepare soldiers for military battle. The link trainer of early 
military pilot training, and now our very complex pilot and 
stewardess training simulations as typified by United Airlines' 
tremendous training station at Denver, Colorado are other examples. 
There seems to be no end to the complexity of the simulations 
which are now being used to prepare personnel in the aviation 
field. Those of you whc have seen the movie. Airport , will 
recognize it, as are most movies, as one big simulation. The 
medical field also makes extensive use of simulated conditions in 
training doctors, nurses and paraprof essional personnel. 

Concepts too remote to the learner can also be taught using 
simulations. Our elementary teachers have, for years, used varia- 
tions of simple simulations to acquaint learners with life in other 
countries. The problems faced by the ghetto child or adult are 
often too remote, too unfamiliar, too repulsive, to our white, 
middle class backgrounds for us to fully understand and take ef- 
fective action to assist correction of the problem. Extensive 
work is now being done to prepare simulation materials for use in 
training teachers, counselors, and administrators to work more 
effectively with the ghetto individual. Some of the most laudible 
work is being done by the University Council for Educational 
Administration located at The Ohio State University where they are 
preparing a number of simulation "packages" in this area. 



package implies a mix of materials which may include: 

I) introductory materials to simulation itself; 2) introductory 
materials to the simulation exercise, instructional objectives 
and modified behaviors of the training package; 3) background data 
to acquaint the student with the actual situation he is dealing 
with including written text, films, slides, tapes, and records; 

4) student exercises to be dealt with by the learner; 5) an in- 
structor's guide for using the materials; and 6) student working 
papers for use in completing the exercises. Note : This general 

composition of a simulation package also holds true, in most in- 
stances, for the simulation packages developed and under develop- 
ment by The Center for Vocational and Technical Education, The 
Ohio State University. 
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Some processes are so complex that their underlying structure 
can be grasped only when trivial details and irrelevant distur- 
bances are eliminated from their study. The complexity of many 
concepts and skills necessary to be learned defy study and under- 
standing in their natural setting. A prime example, and of crucial 
import in today's vocational education picture, is the state and 
local planning process of vocational education. Planning skills, 
techniques, and concepts are, to say the least, difficult to under- 
stand till seen in a working or actual situation and till observed 
and manipulated throughout an entire planning cycle. The Center 
is deeply involved at the present time in developing simulation 
materials to be used in preparing individuals to participate in 
the planning process. 4 

The necessity to learn planning in the context of a complete 
planning cycle brings us to another critical and extremely impor- 
tant feature of simulations. A primary reason for developing 
simulation is that simulations give us the possibility to compress 
time — so that perhaps a one hour long decision-making cycle would 
present one month , one year , or perhaps a number of years in the 
real world. The simulation game or exercise allows participants 
to experiment with different strategies and it forces them to live 
with the consequences of their decisions, not in two months or in 
two years but almost immediately. 

Thus, processes dangerous, remote, or complex can be learned 
through simulation. In addition, the learnings that take place 
from simulations often differ qualitatively from the output of 
other teaching methods. These differences arise out of the struc- 
ture of simulation and the atmosphere generated among those who 
participate in simulation exercises. 

There are some general characteristics of simulations, par- 
ticularly those used in education which should probably be noted. 
Ones which seem important to mention include: 

1. By their nature simulations promote a high degree, .of 
involvement, not only of the learner, but of the in- 
structional staff. 

2. The consequences of one's decisions can be immediately 
observed . 



Center simulation publications' which focus on the planning 
process are Supervision and Decision-Making Skills in Vocational 
Education and Simulation Training in Planning Vocational Education 
Programs and Facilities listed as bibliographical entries 6 and 7. 
An additional package is currently being pilot tested and other 
simulation packages are planned. 
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3. The consequences of decisions arise within one's self, 
generally by one's self and must be immediately dealt 
with. 

4. Simulations alter behavior rather than verbal intentions 
since action with the simulation is based upon behavior 
rather than verbal intentions . 

5. Simulations are self- judging , avoiding reactions to 
authority by the participant. They are generally open- 
ended with no right decision which must be arrived at. 

When simulation exercises are constructed in an open-ended 
fashion, that is, when there is no absolutely correct answer to 
any one exercise, they become especially well suited for use as 
discussion promoting devices. They are exceptionally adaptable 
to the workshop situation and promote the type of discussion where 
an individual can truly learn from his peers. 

Simulation materials are found in a variety of forms and 
degrees of complexity, from the relatively simple in-basket and 
role playing approaches to highly complex gaming and computerized 
packages. Elaborate gaming and simulation techniques have been 
developed for use in industry and the private sector. Public 
education has been slow in moving into the area of simulation. 
However, it does seem that at present a tremendous amount of steam 
is being generated in this regard and educators are moving rapidly 
to provide materials which utilize simulation techniques for 
teacher education and administrator development. 

To conclude this general discussion of what simulation is, I 
would like to emphasize that the simulation techniques or methods 
are nothing more than another teaching tool. The materials will 
be no better than the instructor that uses them — and I would 
caution that some preparation for effectively using simulation 
materials should be obtained by the instructor. A second caution 
should be assumed without my mentioning it. Any simulation mate- 
rial, whether it is developed by The Center for Vocational and 
Technical Education, The University Council for Educational Admin- 
istration, or any other individual or organization, will have to 
be adapted to the situation in which the trainer finds himself 
involved . 

It seems appropriate to discuss briefly a few of the simula- 
tion materials that are available. There are many commercial 
productions of simulation games and simulation materials which I 
will not mention, but I will indicate at least two primary sources 
in the area of education. The University Council for Educational 
Administration, whose headquarters are on The Ohio State Univer- 
sity campus, has developed extensive materials in educational 
administration which use simulation techniques. UCEA generally 
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follows a pattern of contracting with member teacher educators 
and teacher education institutions throughout the country to 
develop these materials. They currently have materials which 
simulate the roles of principal, superintendent, assistant super- 
intendent, finance officer, community college president, curric- 
ulum director, guidance director, and others in secondary and 
post-secondary education. There is also a set of materials, 
connected with UCEA, but handled through the University of Nebras- 
ka, School of Education, which is specifically designed to acquaint 
the secondary school principal with some of the problems and the 
potentials of vocational education. The other source which I will 
discuss is, of course, our own work at The Center for Research and 
Leadership Development in Vocational and Technical Education at 
The Ohio State University. 

The Center began its work in the development of simulation 
materials approximately two years ago as a result of earlier re- 
search and development efforts. Products were developed which 
seemingly had great applicability for improvement of various 
processes in the states, but were not generally used for a variety 
of reasons. One, of course, was communication-dissemination pat- 
terns. Another was that state and local vocational education 
leadership did not know how to effectively utilize the tools that 
were developed. Promising research findings and development ef- 
forts were and are not being implemented due to difficulties en- 
countered in their use and in preparing individuals to use the 
newly developed tools and innovations. The purpose of The Center's 
work in simulation training has been to develop, test, and dis- 
seminate including "training for use" vocational education lead- 
ership training materials which utilize simulation techniques and 
which facilitate the preparation of leadership personnel to use 
the results of research and development findings. 

The development, testing, evaluation, modification and im- 
plementation of simulation materials at The Center is a straight- 
forward yet complex process. As perhaps a typical procedure 
followed in developing simulation materials, The Center's process 
is worthy of description. Figure 2 outlines this process. The 
figure is an adaptation of the steps in the design of an instruc- 
tional simulation system as illustrated in Twelker's article, 
"Designing Simulation Materials. "5 our process closely parallels 
the procedure used by Twelker in the Teaching Research Division, 
Oregon State System of Higher Education. 

The first three steps involve defining the instructional 
problem, describing the operational situation and relating that 



^Paul A. Twelker, "Designing Simulation Systems," Educational 
Technology , October 1969, pp. 64-70. 
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FIGURE ? 

STEPS IN THE DESIGN OF AN INSTRUCTIONAL SIMULATION SYSTEM 
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situation to the instructional problem. Those steps are for the 
most part not unlike other recommended procedures of developing 
institutional materials. Steps four and five involve the speci- 
fication of behavioral objectives and determination of criterion 
by which to measure if the objectives are met. Steps 4 and 5 are 
crucial and often neglected in development work, especially step 
5. Behavioral objectives have at least, become a standard to 
which we give lip service. Not so with criterion measures. All 
too often we specify objectives to be reached without any consid- 
eration for how we will measure their attainment or even check 
our progress toward their attainment. Writers of our simulation 
materials must develop criteria which they will accept as valid 
measurements of objective attainment. 

Steps 6, 7, and 8 move from general instructional material 
development to simulation development. All instructional problems 
certainly are not best solved through simulation techniques. Only 
those problems with basis in the real world and for which models 
or representations can be developed, can be simulated. The type 
of simulation required must be determined and the experiences to 
be simulated identified and described. 

Steps 9-11 follow a logical sequence of development, testing, 
modification and dissemination. The package is developed and tried 
out in a controlled pilot test. After evaluation and improvement 
more extensive field trials, generally taking the form of a demon- 
stration-evaluation-training for use workshop, takes place . ° After 
further modification, if needed, our materials are disseminated 
and made available to the field. 

The Center has completed two projects which have resulted in 
simulation training packages for use with the vocational education 
leadership personnel. These materials are adaptable to both in- 
service and preservice activities and will be made available to 
individuals and institutions in the states who can utilize them in 
their personnel development programs. 

The two packages of materials currently available are: Re- 

search 51, Supervision and Decision-Making Skills in Vocational 
Education: A Training Program Utilizing Simulation Techniques ; 



u Demonstrat i on-Eva I uat i on-Tra i n i ng for Use Workshops are 
arranged for by The Center's Dissemination Specialist. The peri- 
odic meetings are designed to demonstrate the use of Center prod- 
ucts to potential user groups. Simulation materials particularly 
rely upon the training for use component of the workshops. It is 
felt that due to difficulties sometimes encountered in their use, 
individuals wishing to use simulation materials should be prepared 
to effectively use them. 
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and Research 52 3 Simulation Training in Planning Vocational Edu- 
cation on Programs and Facilities . These have previously been 
noted and a bibliographical reference provided. Research 51 
emphasizes the personal and technical skills involved in super- 
vision and Research 52 focuses upon skills needed by leadership 
personnel in the planning of vocational education programs with 
the emphasis upon facility planning. 

A third simulation package is currently being pilot tested 
and should be completed and available for use by late fall of 
this year. It will focus upon the state planning function in 
vocational education. The current materials have been designed 
primarily for state leadership personnel of vocational education. 
However, the materials for the most part are usable in their 
current form for secondary and post-secondary vocational education 
administrator training and with normal adaptive processes should 
be quite readily usable with a large number of population groups. 
The Center anticipates the funding and initiation this fall of a 
project which will develop at least two additional packages of 
simulation materials. 

In reference to publications R-51 and R-52, the materials in 
their present booklet form are not usable as classroom documents. 
Teacher educators or personnel development specialists who will be 
conducting in-service or preservice training using these materials 
will find it necessary to reproduce portions of the publications 
in classroom quantities. These would include the student exer- 
cises, data bank, and the working papers. Permission to reproduce 
is of course not needed since the material is under public domain. 

In conclusion, simulation is a proven instructional method 
and tool which exhibits great promise for vocational education 
applications. Utilization of the materials presently available 
and under development can provide a valuable assist in the devel- 
opment of vocational education leadership. The Center invites 
your use, critique, evaluation, and feedback regarding our mate- 
rials . 
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"THE DELPHI TECHNIQUE! A TOOL FOR INQUIRY" 
June 3, 1970 

KENNEY E. GRAY 



The DELPHI Technique is a means of securing expert convergent 
opinion without bringing the experts together in face-to-face 
confrontation. This convergent opinion of experts is usually 
gained through the use of successive questionnaires and feedback 
with each round of questions being designed to produce more care- 
fully considered group opinions. The DELPHI Technique in its 
simplest form: 

. . .eliminates committee activity among the experts 
altogether and replaces it with a carefully designed 
program of individual interrogations (usually best 
conducted by a questionnaire) interspersed with in- 
formation input and opinion feedback. I 

Helmer and his colleagues at the Rand Corporation developed 
the DELPHI Technique in the early lS50's to obtain group opinions 
about urgent defense problems. The technique has also been used 
for predicting long-range developments in a scientific-technological 
area including defense, automation, and space research. It is used 
in identifying both majority and minority opinions as a basis for 
formulating goals and setting priorities. ^ An understanding of 
the DELPHI Technique can be enhanced by reviewing an example of 
the use of the technique. Helmer cited the following example in 
an effort to describe the procedure: 

When inquiring into the future of automation, each 
member of a panel of experts in this field was asked 
to estimate the year when a machine would become 
available that would comprehend standard IQ tests 
and score above 150 (where "comprehend 11 was inter- 
preted behavioristically as the ability to respond 
to printed questions possibly accompanied by dia- 
grams) . The initial responses consisted of a set 
of estimates spread over a sizeable time-interval. 
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John Pfeiffer, New Look at Education : Systems Analysis in 
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